With the emergence of multicore processing architectures and parallel processing arrays on a single die, the efficiency and throughput of on-chip can become a bottleneck that can keep a system on a chip from reaching a peak theoretical performance. Shared bus architectures, such as the AMBA bus and AXI bus, have limited scalability and cannot be used to interconnect more than a dozen masters without suffering from size and timing penalties. With the number of processing elements increasing, a traditional shared bus network might not be sufficient.
Work has been done in creating Networks on a Chip (NOC) to improve the scalability and performance of on-chip networks. Some interconnection architectures that have been proposed include torus networks, mesh networks, and tree networks. Much of the detailed work on NOCs has borrowed concepts from traditional packet based systems. To improve data routing efficiency and prevent deadlocks in the network, sophisticated techniques have been employed, including packetizing of data, breaking packets into sub-packets called flits, allowing for worm hole routing, and inserting FIFO buffering at each interconnection node in the network.
Three dimensional networks are already employed in real consumer products. Many cell-phones are using stacked chip technology. IBM recently announced silicon through via technology that is close to production.
The efficiency of an NOC is dependent on a number of different factors, such as network topology, routing algorithms, packet strategy, buffer sizes, flow control, and quality of service support.
The network topology defines the layout of the wires that connects nodes in the network together. Some common topologies include mesh, torus, binary tree, hierarchical, honeycomb, and rings. The ideal topology will depend on the application at hand and the technology used as an interconnection fabric. NOCs for example have very different cost trade offs from interchip networks. For interchip networks, wires and pins can be expensive and should be minimized, whereas in intrachip networks, wires are almost free.
The routing algorithm determines how data is routed from the source to the destination. Paths can be set up at the source or done in a distributed fashion at each node. A well designed routing algorithm will minimize traffic hotspots to improve the throughput of the network without adding significant complexity.
Flow control dictates the movement of packets along the network using at least one stage of buffering at each routing node. When a downstream buffer is unavailable, backwards pressure is asserted to stop transactions from proceeding. Some common methods for creating backwards pressure are: credit based, on/off, and ack/nack buffer flow control.
A large portion of NOC area and power is dedicated to buffering so choosing the correct buffer size is key to achieving a well balanced network. Large buffers allow the network to operate closer to peak capacity by temporarily storing data in buffers when there is link contention. As long as the pressure on the local link goes away before the buffer fills up, throughput is not compromised. Buffers are thus especially effective for random traffic that is not overly bursty.